An In-Place Algorithm for Irregular All-to-All Communication with Limited Memory

In this article, we propose an in-place algorithm for irregular all-to-all communication corresponding to the MPI Alltoallv operation. This in-place algorithm uses a single message buffer and replaces the outgoing messages with the incoming messages. In comparison to existing support for in-place communication in MPI, the proposed algorithm for MPI Alltoallv has no restriction on the message sizes and displacements. The algorithm requires memory whose size does not depend on the message sizes. Additional memory of arbitrary size can be used to improve its performance. Performance results for a Blue Gene/P system are shown to demonstrate the performance of the approach.